1. Field of the Invention
The present invention relates to an image processing apparatus, an image processing method, and an image capturing apparatus, and in particular relates to an image processing apparatus, an image processing method, and an image capturing apparatus that can detect a specific subject such as a person, an animal, or an object included in an image, or a part of the subject.
2. Description of the Related Art
An image processing technique of automatically detecting a specific subject from an image is highly useful, and can be utilized, for example, for specifying a face region of a person in a moving image. Such an image processing technique can be employed in many fields including teleconferences, man-machine interfaces, security systems, monitor systems for tracking human faces, and image compression. A digital camera or digital video camera that detects a face from a captured image and optimizes focus or exposure for the detected face is also known. Japanese Patent Laid-Open No. 2005-318554 discloses an apparatus that focuses on a face detected from an image and captures an image at an optimum exposure for the face.
Furthermore, there is a known image capturing apparatus that has a function of selecting a subject that serves as a main subject from a plurality of subjects, in order to perform more accurate AF (Autofocus) and the like even in the case where a plurality of specific subjects are present in an image.
Japanese Patent Laid-Open No. 2002-51255 discloses an image capturing apparatus that selects a subject that serves as a main subject based on the states of the subjects present in an image. A state of a subject refers to various parameters indicating, for example, a distance from the image capturing apparatus to the subject, an area of the subject in the image, a position of the subject in the image, a distance between subjects, and the like. Moreover, Japanese Patent Laid-Open No. 2008-5438 discloses an image capturing apparatus that suppresses frequent switching of a main subject, by selecting a current main subject in consideration of information about a previously selected main subject.
One method for selecting a main subject from a plurality of subjects according to a conventional technique is a method of performing a main subject selection process based only on the states of subjects at a specific time. This method has the advantage of enabling the appropriate main subject at the specific time to be selected, but also has the problem of causing frequent switching of the main subject between a plurality of subjects.
This problem is described below, with reference to FIG. 1. In an example shown in FIG. 1, subjects 100 and 101 are each detected by an image capturing apparatus in frames #1 to #4 of a moving image in chronological order. In FIG. 1, a main subject is indicated by a solid line box, and a subject other than the main subject is indicated by a dashed line box.
As an example, the main subject is determined according to a priority that is computed based on a distance from the image center to the gravity center of a subject image and a size of the subject image, in the states of the detected subjects 100 and 101. Suppose the distance from the image center to the subject gravity center and the subject size, which indicate the state of the subject, are similar between the two subjects 100 and 101, as shown in FIG. 1. In such a case, slight changes in the states of the two subjects 100 and 101 cause frequent switching of the main subject between the subjects 100 and 101. That is, priority rankings of the detection results change.
In the case of performing AF and the like on the main subject, when the main subject is frequently switched between a plurality of subjects, such changes of the main subject cause an AF lens and the like to be frequently driven. This induces a possibility of reducing the operational stability and leading to a loss of accuracy of focusing and the like. In addition, in the case where a marker or the like indicating the main subject is displayed in the display of the image capturing apparatus, such frequent changes of the marker display as shown in frames #1 to #4 in FIG. 1 are annoying to the user.
Another method for selecting a main subject from a plurality of subjects is a method of performing a main subject selection process based on the states of subjects at a specific time and information on a previously selected main subject. This method has the advantage of suppressing frequent changes of the main subject, but also has the problem of making it difficult to switch the main subject even in a scene where the main subject needs to be switched.
This problem is described below, with reference to FIG. 2. In an example shown in FIG. 2, subjects 102 and 103 are each detected by an image capturing apparatus in frames #1 to #4 of a moving image in chronological order. In FIG. 2, a main subject is indicated by a solid line box, and a subject other than the main subject is indicated by a dashed line box.
As an example, the main subject is determined according to a priority that is computed based on a distance from the image center to the gravity center of a subject image, a size of the subject image, and a priority ranking in a previous frame, in the states of the detected subjects 102 and 103. To suppress frequent switching of the main subject, it is necessary to increase, in the priority computation, the influence of the priority ranking in the previous frame to a certain extent.
In detail, priority rankings of the subjects 102 and 103 in frame #1 in FIG. 2 are determined first. In this example, the subject 102 is determined to have a higher priority ranking than the subject 103, so that the subject 102 is selected as the main subject. Once the main subject has been determined, in the following frames #2 to #4 a situation where it is difficult to switch the main subject arises, due to the influence of the priority rankings in frame #1.
If the main subject is not switched from the subject 102 to the subject 103 even when the states of the subjects in the captured image are as shown in frame #4 in FIG. 2, the user may feel something is wrong.